System for automatically recognizing plants

ABSTRACT

The disclosure relates to a system for automatic recognition of plants, including a central server unit, a network, an automatic picture processing unit, a plant database unit, in which plant data are stored, a picture database unit, in which picture data related to plants are stored, and a mobile device with a camera, the camera of the mobile device configured to provide picture data of a plant and the picture data transmitted via the network to the server unit, the automatic picture processing unit together with the plant database unit and the picture database unit on the basis of the transmitted picture data is configured to perform a comparison with data in the plant database unit and the picture database unit and the transmitted picture data of the plant such, that the plant is defined and the definition can be is transmitted back to the mobile device.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a U.S. National Phase of International PatentApplication Serial No. PCT/EP2017/054318 entitled “SYSTEM FORAUTOMATICALLY RECOGNIZING PLANTS,” filed on Feb. 24, 2017. InternationalPatent Application Serial No. PCT/EP2017/054318 claims priority toGerman Patent Application No. 20 2016 004 430.4 filed on Jul. 20, 2016.The entire contents of each of the above-referenced applications arehereby incorporated by reference for all purposes.

TECHNICAL FIELD

The present invention relates to a system for automatic recognition ofplants.

BACKGROUND AND SUMMARY

In the field of agriculture, but also in the field of botany andeducation in connection with professions related to agriculture orforestry, but also in the field of biology and botany, it is necessaryto recognize plants accurately.

From DE 43 29 343 A1 is already known a field useable picture analyticmethod for automatic recognition and removing of weeds. Here, acomputing device of a camera is installed onside, to recognize weeds.

However, only a certain amount of weeds can be detected and furthermore,the system according to DE 43 29 343 A1 is relatively large andtherefore cannot be handled very well.

From DE 10 2010 048 298 A1 an installation of the detection of growth ofa plurality of biological probes is known. This system is especiallyrelated to do the phenotyping and the characterization of plants. Bymeans of this installation, the growth of plants will be detected. Theseplants will be arranged in a shelf. A camera is mounted to a movingdevice and will be moved for several periods of time along the samemoving path. By this, several single pictures will be put together as amosaic from the plurality of samples, which will then be stored in adatabase and be shown as a mosaic graphic.

From DE 102 45 900 A1 a picture based request system for searchingmachines for mobile devices with an integrated camera is known. By meansof this system, objects can be recognized automatically. However, thesystem relates to the recognition of sights of a city or to therecognition of plates, which have a text in another language, which theuser cannot speak. Other parts of use are city guides or museum guidesand/or face recognition systems.

From DE 101 10 979 A1 is known a system and arrangement for linking ofoptical recognized patterns with information. This arrangement is usedfor linking of optically detected patterns of an object with informationwith a camera for detecting the pattern, further being equipped with apattern storage, in which reference patterns may be stored, with anediting unit, which is comparing the reference pattern with the detectedpatterns and with an information storage, in which the information inrelation to the reference pattern is stored, with a further module,which is processing the information. Main applications, which are namedin this application, are universal remote, controllers in connectionwith CD players or TVs.

It is an object of the present invention to provide a system forautomatic recognition of plants, especially in that that plants can beeasily and accurately be recognized and that an accurate recognitionwith at the same time immediate workflow, i.e. recognition anddefinition within seconds or minutes is possible.

This object is solved according to the present invention with a systemfor automatic recognition of plants. Accordingly, a system for automaticrecognition of plants includes at least one central server unit, atleast one network, at least one automatic picture processing unit, atleast one plant database unit, in which plant data are stored, at leastone picture database unit, in which picture data related to plants arestored, and with at least one mobile device with at least one camera,wherein the central server unit is in use via the network with themobile device, wherein by means of the camera of the mobile devicepicture data of a plant can be provided and transmitted via the networkto the server unit, wherein the automatic picture processing unit iscapable together with the plant database unit and the picture databaseunit on the basis of the transmitted picture data to perform acomparison with data in the plant database unit and the picture databaseunit and the transmitted picture data of the plant and that they areanalyzed such, that the plant can be defined and the definition can betransmitted back to the mobile device via the network by means of theserver unit.

The invention is based on the basic idea that the system for automaticrecognition of plants is provided, wherein for the definition andrecognition of the plant it is necessary to take a picture with themobile device of the plant in any growth state of the plant. Thispicture will then be transmitted to the server unit and there by meansof an automatic picture processing unit processed, automaticallyrecognized and by means of a comparison with a database, in which plantdata and a further database, in which the raw picture data of therespective plant are stored, compared.

By means of this comparison it is possible to automatically check theplant being depicted in the picture to recognize automatically the plantbeing depicted on the picture and to transfer this information back tothe mobile device.

Furthermore, it is possible that a plant protection product storage unitis provided, in which plant protection product data are stored, whereinfurther in the plant protection product storage unit linking data arestored, with which the plant protection product data with the picturedata and/or plant data by means of a data analysis module can be linked.It is possible in this regard that the plant data and also the picturedata are so called weed data or related to weeds. In particular, in theagricultural or forestry area it may be of interest to define plants inany phase of growth and at any time of the year alone on the basis ofpicture data and based on this definition also receive a proposal, whichplant protection product, especially pesticide can be used or should beused to remove unwanted plants.

Furthermore, it is possible that the plant protection product datainclude substance data and/or manufacturer data related to themanufacturer of the plant protection product. By providing the activesubstance data, it is possible to provide information by means of thesystem related to the plant protection product to be used and its activesubstances and active substance groups, so that the selection of thesuitable plant protection product(s) is simplified. It is especiallypossible that a preference with respect to a specific manufacturer of aplant protection product is possible. Thereby, it is easier for the userof the system to do a pre-selection so that he gets a selection ofavailable plant protection products and he does not have to scrollthrough a plurality of options and then has to select there again.

Moreover, there may be a filter module, which is capable and/or arrangedsuch that only manufacturing data of a specific manufacturer of a plantprotection product is possible for linking the plant protection datawith the picture data and/or the plant data by means of the analysismodule. By this pre-selection the handling of the overall system can besimplified. Furthermore, by means of the pre-selection of themanufacturing data also the procedure of data requests can be simplifiedand made faster. In particular, it is possible that loss data must betransferred and transmitted and also in a search must be done in asmaller data volume. By this, the request time and also the data volumeto be transmitted is reduced.

Furthermore, the picture data may comprise location data, such as GPSdata. By linking the mere picture data with location data it becomespossible that the existence of specific species of plants can be linkedwith location data and thus, this kind of information can be then usedover the whole area, which is used for agriculture or forestry. It isalso possible that in connection with the extinction of weeds suitableweed destruction products can be used at the right location and thus,the process of distribution is simplified.

Moreover, the picture data may additionally comprise date data. By thedate data it is possible to make a prediction and analysis to the growthstate of the plant data being enclosed in the picture data. Depending onthe time of the year and the date a plant has different growth states,which can then better be brought in context. By this, also the time,which is necessary for a databased request and for an analysis, can bereduced. It is possible that in the plant data module the stored dataare sorted depending on the time of the year or that a selection can bedone based on the time of the year.

The picture data can also comprise time of the day data. By suchinformation it becomes possible that light and shadow effects can betaken into account and doing the analysis. By this, the quality of theanalysis can be improved.

Furthermore, the mobile device may be a mobile phone, especially asmartphone. The smartphone may be especially a smartphone with a cameraand respective link or links to the Internet.

Furthermore, it is possible that the mobile device is a tablet computer.Especially, the tablet computer can be tablet with a camera function andlink to the Internet.

It is also possible that furthermore a plant disease database unit isprovided in which plant disease data are stored, furthermore, wherein inthe plant disease database unit link data are stored, with which theplant disease data can be linked with picture data and/or plant data bymeans of the data analysis module. It is especially possible that on thebasis of recognized plant diseases, which may be detected automaticallyby the system, especially on the basis of color, structure and forexample changes in color or changes in structure of the plant a suitableplant protection product can be selected. It is also possible that onlythe plant disease is detected automatically to come to conclusions forfurther measurements in the agricultural or forestry area.

It is also possible that furthermore a plant pest database unit isprovided in which plant pest data are stored, wherein in the plant pestdatabase unit linking data are stored, by means of which a plant pestdata can be linked with picture data and/or plant data by means of thedata analysis module. By this, the advantages achieved that by taking apicture of a plant and the respective and related picture data alsoplant pests can be recognized such as aphid, beetles, fleas, insects oralso eating patterns, which can then be compared with information datastored within the system. The user of the system can be provided withrespective information, which kind of plant pests are present.Furthermore, it is possible that the user receives automaticallyinformation by the system, which kind of plant protection product shouldbe used in connection with the recognized plant pests.

BRIEF DESCRIPTION OF THE DRAWINGS

Further details and advantages of the invention shall now be describedwith an embodiment in connection and as shown in the drawing.

The only FIGURE (FIG. 1) shows schematically the general layout of thepresent invention, especially a possible embodiment of a system forautomatic detection of plants.

DETAILED DESCRIPTION

The only drawing (FIG. 1) shows a system 10 for automatic recognition ofplants.

The system 10 comprises a central server unit 12.

Furthermore, there is a network 14, here the Internet, by means of whichthe central server unit 12 may be in data exchange with other devices.

The server unit 12 comprises an automatic picture processing unit 16, aplant database unit 18, a picture database unit 20, a plant protectionproduct database 22 and a filter module 24. The server unit 12 comprisesa data analysis module 30.

In the plant database unit 18 the plant data are stored.

In the picture database unit 20 picture data related to plants arestored, typically related to plants, which are also stored in the plantdatabase unit 18.

The picture data related to plants are plant picture data for the wholecircle of the year related to plants in all kind of growth stages andembodiments.

The plant protection product database unit 22 is configured such that init plant protection data are stored and furthermore, also linking dataare stored, by means of which the plant protection product data can belinked with picture data and plant data via the data analysis module 30.

Furthermore, also the manufacturing data related to the manufacturer ofa plant protection product is part of the plant protection product data.

The filter module 24 is configured such that by means of a filter module24 it can be preset or selected that only manufacturer data of aspecific manufacturer of a plant protection product can be used for thelinking of the plant protection product data with the picture data andthe plant data by means of the data analysis module 30.

By means of the data analysis module 30 linking data, which has storedin the plant protection product database 22, can be linked with picturedata and plant data.

Furthermore, there is a plant disease database unit 32, in which plantdiseases data are stored. Furthermore, in this plant disease protectiondatabase unit 32 linking data are stored, by means of which plantdisease data can be linked with picture data and plant data by means ofthe data analysis module 30.

Additionally, there is a plant pest database unit 34, in which plantpest data are stored. In this plant pest database unit 34 linking dataare stored by means of which plant pest data can be linked with thepicture data and plant data by means of the data analysis module 30.

The picture data can comprise further location data, here GPS data, datedata and also timing data, i.e. a so-called time stamp.

The system 10 comprises mobile devices such as a smartphone 26 or atablet pc 28. The smartphone 26 can be for example an iPhone and thetable pc can be for example an iPad.

The function of the system 10 can be described as follows:

By means of the smartphone 26 or the tablet pc 28 the user can take apicture on an agricultural area or a forestry area, wherein the pictureis a picture of the plant to be analyzed.

This can be done by means of an app or directly by means of the camerafunction. The picture is then also attached with the location data andwith a time stamp and this kind of information will then be transmittedvia the network 14, i.e. the mobile communication network and theInternet to the central server unit 12.

The central server unit 12 comprises data input and output means, bymeans of which data can be received, but also transmitted back to themobile devices.

In the central server unit 12 an analysis of the transmitted pictures isperformed by means of the automatic picture processing unit 16 tocompare the information contained in the picture data related to a planton the basis of in the picture database unit 20 contained picture dataand in connection with the plant database unit 18 to find out therespective plant automatically.

After the recognition of the plant by means of the data analysis module30 it is possible to select automatically on the basis of the plantprotection product data contained in a plant protection product databaseunit 22 a proposal, which weed killer or plant protection product shouldbe used.

On the basis of the transmitted picture data the data analysis module 30may also recognize on the basis of the plant disease database unit 32whether and which plant diseases are present. Here, especially on thebasis of color and/or structure or the specific amendments it can bedetected, whether or which plant disease is present. This process alsois done automatically.

On the basis of picture data acquired, which are checked against eachother, it is also possible to detect, whether plant pests are present.Thereto, in the plant pest database 34 the stored data are used andthose data are then linked with further present data. For example it ispossible that the plant protection product data, which are stored in theplant protection product database 22 are used to provide a proposal,which plant protection product, i.e. which pest killer can be used. Theanalysis is based on the fact, whether on the transmitted picturesaphids, beetles, fleas, insects or specific eating patterns arerecognizable.

The automatic picture analysis, which is done with the automatic pictureprocessing unit 16 is oriented on reference points or referencestructures on the transmitted pictures and the information within thetransmitted pictures related to the plants from which a picture istaken.

It is also possible that already processed data can also be used that onthe basis of pictures which have been transmitted to the central serverunit 12, such picture information will then also be stored in the plantdatabase unit 18. It is possible that this will happen only then, when aspecific authorization of a qualified user has happened. Such a user maybe a power-user or an employee of the system owner, whereas such userhas specific knowledge with respect to plants.

In connection with plant diseases or the presence of pests, it can bepossible that the specific location data can be linked with informationregarding plant diseases or the presence of plant pests.

Form the information regarding location and the presence of plantdiseases, pests, but also from the presence of weeds, it is possible tofind out, where they are present and this information can then beforwarded. By this, the use of plant protection products can besignificantly be reduced, as only the areas, which have been identifiedas the once with diseases or pests, can then be treated with therespective plant protection products or pest killers.

In this regard, it is possible that the system 10 for automaticrecognition of plants is mounted on a plant protection product sprayingmachine, which is mounted or attached to an agricultural vehicle suchthat by working on an agricultural area, where plant protection productsare needed, the plant protection product or the pest killer will then besprayed out targeted to the respective area, where such products areneeded.

It is also possible that the system 10 comprises a chat functionality,which is established between the central server unit 12 and a pluralityof mobile devices 26, 28 or also comprises an expert call-center or anexpert central with a chat functionality such that the users with themobile devices 26, 28 may communicate with the experts or even betweeneach other regarding the pictures that have been taken with the mobiledevices 26, 28, whereas the pictures are related to plants, pests orplant diseases.

The information may be transmitted back by the server unit 12 to themobile devices 26 or 28.

Also it is possible that by means of the filter module 24, either set bythe user or preset by the manufacturer, only plant protection productdata of a specific manufacturer will be transmitted back in connectionwith the recognized plant.

Generally speaking, it is also possible that after a deactivation of afilter module 24 or by respective settings of the filter module 24 allpossible plant protection products or pesticides or weed killers can besearched and transmitted back by the server unit 12 via the network 14to a mobile device 26 or 28.

REFERENCES

-   10 System-   12 Server unit-   14 Network-   16 Picture processing unit-   18 Plant database unit-   20 Picture database unit-   22 Plant protection product database-   24 Filter module-   26 Smartphone-   28 Tablet-PC-   30 Data analysis module-   32 Plant disease database unit-   34 Plant pest database unit

The invention claimed is:
 1. A system for automatic recognition ofplants, comprising: a central server unit; a network; an automaticpicture processing unit; a plant database unit, in which plant data arestored; a picture database unit, in which picture data related to plantsare stored; and a mobile device with a camera, the central server unitconfigured to be in use via the network with the mobile device, thecamera of the mobile device configured to provide picture data of aplant and the picture data transmitted via the network to the serverunit, wherein the picture data also comprise date data, the automaticpicture processing unit configured to, on the basis of the transmittedpicture data and date data and together with the plant database unit andthe picture database unit, perform a comparison with data in the plantdatabase unit and the picture database unit and the transmitted picturedata of the plant such that the plant is defined and the definition istransmitted back to the mobile device via the network by the serverunit.
 2. The system according to claim 1, further comprising a plantprotection product database unit in which plant protection product dataare stored, wherein further in the plant protection product databaseunit linking data are stored, by which the plant protection product datais linked with the picture data and/or plant data by a data analysismodule.
 3. The system according to claim 2, wherein the plant protectionproduct data comprise active substance data and/or manufacturer datarelated to a manufacturer of a plant protection product.
 4. The systemaccording to claim 3, further comprising a filter module configured topreset and/or select that only manufacturing data of a specificmanufacturer of a plant protection product is to be used for the linkingof plant protection data with the picture data and/or the plant data bythe data analysis module.
 5. The system according to claim 1, whereinthe picture data also comprise location data.
 6. The system according toclaim 1, wherein the picture data also comprise time data.
 7. The systemaccording to claim 1, wherein the mobile device is a mobile phone. 8.The system according to claim 1, wherein the mobile device is a tabletpc.
 9. The system according to claim 1, further comprising a plantdisease database unit, in which plant disease data are stored, whereinfurther in the plant disease database unit linking data are stored, bywhich the plant disease data is linked with picture data and/or plantdata by a data analysis module.
 10. The system according to claim 1,further comprising a plant pest database unit, in which plant pest dataare stored, wherein furthermore in the plant pest database unit linkingdata are stored, by which a plant pest data is linked with picture dataand/or plant data by a data analysis module.